Warping for image stabilisation

Introduction and overview With a see-through augmented reality (AR) display the image in the display should be updated within a few milliseconds after a head movement of the observer, in order to maintain the virtual objects at a stable location in the visible world. In other words, the virtual camera has to mimic the head movements of the observer as close as possible. Allowed lags are in the order of 10 ms (Pasman, 1997; Azuma and Bishop, 1994). These constraints are difficult to meet, even with existing very fast polygon rendering engines. Optimised polygon rendering on high-performance Silicon Graphics machines still gives total lags of about 30 ms wireless AR display we should also minimise the required computing power. To solve these problems, a previously rendered image can be deformed (warped) to approximate a full rerendering at the new camera position. Appendix A describes different types of warping. As warping may be done 10-20 times faster than a complete rerendering (Torborg and Kajiya, 1996), this may resolve the lag problems. Ideally the warp is controlled by control points in 3D space, in order to find a warp that brings the image as close as possible to the completely rendered image. But approximations may be used to avoid costly perspective projections. To generate warped images that mimic the effect of a camera rotation, or to simulate camera rotation (Figure 1), the rendered image has to be shifted and/or rotated as a whole. To simulate camera translations a more complex transformation is required: parts in the image depicting more distant objects have to shift relatively less than objects closer to the camera. Therefore, simulating a camera translation requires more sophisticated techniques than simulating camera rotation.